In a large-scale distributed storage system, individual computing nodes will commonly fail or become unavailable from time to time. Therefore, storage systems typically implement some type of recovery scheme for recovering data that has been lost, degraded, or otherwise compromised due to node failure or other causes. One such scheme is known as erasure coding. Erasure coding generally involves the creation of codes used to introduce data redundancies (also called “parity data”) that is stored along with original data (also referred to as “systematic data”) to thereby encode the data in a prescribed manner. If any systematic data or parity data becomes compromised, such data can be recovered through a series of mathematical calculations.
At a basic level, erasure coding for a storage system involves splitting a data file of size M into X chunks, each of the same size M/X. An erasure code is then applied to each of the X chunks to form A encoded data chunks, which again each have the size M/X. The effective size of the data is A×M/X, which means the original data file M has been expanded A/X times, with the condition that A≧X. As a consequence of various encodings, any Z chunks of the available A encoded data chunks (where A≧Z≧X) can be used to recreate the original data file M. Erasure codes where Z=X are referred to as maximum distance separable (“MDS”), and are considered optimal as they have the greatest error correcting and detecting capabilities.
A simple example of a (4, 2) erasure code applied to a data file M is shown in FIG. 1. As shown, a data file M is split into two chunks X1, X2 of equal size, and an encoding scheme is applied to those chunks to produce four encoded chunks A1, A2, A3, A4. By way of example, the encoding scheme may be one that results in the following relationships: A1=X1;A2=X2; A3=X1+X2; and A4=X1+2×X2. In this manner, the four encoded data chunks can be stored across a storage network, such that the one encoded data chunk is stored in each of four computing nodes. Then, the encoded data chunks stored in any two of the four computing nodes can be used to recover the entire original data file M. This means that the original data file M can be recovered if any two of the computing nodes fail.
Erasure-coded data stored in a distributed data storage environment (e.g., a NetApp® StorageGRID® system) can span multiple computing nodes. Operations for restoring erasure-coded data may involve large data transfers among computing nodes. For example, successfully repairing erasure-coded data stored on some or all of a computing node or volume may involve transferring one or more large data sets from one or more volumes on source nodes to one or more volumes on destination nodes.
Thus, if erasure-coded data is stored across multiple sites, a significant data loss may require transferring substantial amounts of data over a wide-area network (“WAN”). For example, erasure-coded data may be stored across three sites. Content at one of the sites may be lost due to a failure of a computing node at that site. Restoring the lost content may require transferring erasure-coded data from the two remaining sites to the third site via a WAN. In this erasure-coding scheme, the amount of data transferred over the WAN is twice the amount of lost data to be restored. To generalize, for simple one-to-one mappings of erasure-coded chunks to nodes, the amount of data required to rebuild a given node is X times the size of the data lost on that node. Thus, as the size of X increases, the cost of a rebuild also increases. This transfer process may present significant expenses in terms of network bandwidth, processing resources, storage resources, or some combination thereof.
Therefore, alternatives to using a WAN for transferring data during a restoration process are desirable.